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We consider backward stochastic differential equations (BSDEs) re- 
lated to finite state, continuous time Markov chains. We show that ap- 
propriate solutions exist for arbitrary terminal conditions, and are unique 
up to sets of measure zero. We do not require the generating functions 
to be monotonic, instead using only an appropriate Lipschitz continuity 
condition. 

1 Introduction 

Consider a continuous time, finite state Markov chain X = {X t , t £ [0, T]}. We 
identify the states of this process with the unit vectors in Mr , where N is 
the number of states of the chain. 

We consider stochastic processes defined on the filtered probability space (f2, 
J 7 , {J-t}, P), where {Ft} is the completed natural filtration generated by the 
cr-fields T t = a({X u ,u <(},Fef T : P(-F) = 0), and T = T T - Note that, as 
X is right-continuous, this filtration is right-continuous. If A t denotes the rate 
matrix for X at time t, then this chain has the representation 
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where M t is a martingale. (See Appendix B of [1].) 



For an Ft measurable, R valued, P-square integrable random variable Q, 
we shall discuss equations of the form 

Z t + I F(uj,u,Z u ,Y u )du+ I [G{cj,u-,Z u _) + Y u -]dM t = Q (2) 

J]t,T] J]t,T] 

for functions F : Qx[0,T}xR K x«. KxN -> R K and G : fix [0, T] xR K -> M KxJv . 
These functions are assumed to be progressively measurable, i.e. F(.,t, Zt,Y t ) 
and G(., t, Z t ) are jT t measurable for all t 6 [0, T]. 

We seek a solution of (J2J) , that is a pair (Z, y) , where Z is an R K valued 
adapted process and Y is an R KxN adapted process. We shall address this 
in four stages: firstly we shall show a martingale representation theorem in 
this framework, then we shall use this to show the existence and uniqueness of 
solutions in three stages of increasing complexity in F and G. This is essentially 
the same approach as in [7] ; however the details are not the same as our dynamics 
differ. The key result presented here is Theorem l6.21 an existence and uniqueness 
result. This result can be seen as a special case of that obtained by [2]; however 
we here present explicit formulae for certain quantities of which they assume 
the existence, and, by establishing a martingale representation theorem, do not 
require the existence of a 'non-hedgeable' process. 

As a side note, observe that @ is equivalent to 

Z t + [ F*(cj : u,Z u ,Y u )du+ [ [G(LJ,u-,Z u -)+Y u _(Lj)]dX t = Q 

J]t,T] J]t,T] 

where F*(w, u, Z u , Y u ) = F(uj, u, Z u , Y u ) - [G{uj, u- Z u -) + Y u -{u))]A u X u {uj). 



2 Preliminary Concerns 

We note that the optional quadratic variation of M t is given by the matrix 
process 

[M,M] t = ]T AM U AM*. 

0<u<t 

Recalling that A is the rate matrix of the Markov chain X, the predictable 
quadratic variation is 

(M, M) t = I [diag^Xu) - diag(X„)vl* - A u disig( X u )}du. 
J]o,t] 

This can be seen by considering 

diagpr t ) = diag(Xo) + / dia,g(A u X u ) du + diag(dM„) 

J]0,t] J]0,T] 
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and 

diag(X t ) = X t X: 



X Q X*+ f X u X* u A* u du+ ( X u _dM* u + [ A u X u X* u du 

J]0,t] J]0,t] J]0,t] 

+ f dM u x:_ + am„am: 

J ]0.t] n ,„ <f 



Equating these two gives 



[M, M] t = L t + [diag{A u X u ) - diag(X u )^; - A u diag(X u )]dv 
J]o,t] 



for some martingale L. This in turn implies that 

(M, M) t = / [dia,g{A u X u ) - diag(X u )A* u - A u diag{X u )}du 

J]0,t] 

as desired. 

Define the following quantities: 

(C,D) = Tr(CT>*), 
||C|| 2 = (C,C), 

(C,D) V = Tr(C[dmg(A u V) - dmg(V)A* u - A u diag(V)]D*), 

\\C\\v = (C) 

where V is a (basis) vector in M. N . As the matrix 

dmg(A u V) - diag(V)A; - A M diag(V) 

is symmetric and positive (semi-)definite, this is a well defined (semi-)norm. 
One notable feature of this notation is that 

\\Cf Xu du= f Tv(Cd(M,M) u C*)- 

t,T] J]t,T] 

Note that for any adapted process C, the quantity ||C||^ = {||C U ||^ , u e [0, T]} 
is an adapted random process with values in [0, oo[. Hence its expectation is 
well defined for each u, 

E\\C u \\ 2 Xu = E[Tr(C u [diag(A u X u ) - diag(X„)^ - A u dmg(X u )]C:)}. 

We restrict our attention to when 

E\\Z U \\ 2 < +00 

for all u and 

E\\Y u f Xu < 
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for almost all u (i.e. < +00 d(M, M) u -&.s.). This, coupled with the 

Lipschitz conditions placed on F and G will immediately imply 

E \\F(aj,u,Z u ,Y u )\\ 2 du < +00 
J]a.T] 

and 

E { \\G(u),u,Z u )\\ 2 du < +00. 

J]0,T] 

This assumption has proven to be important when dealing with similar equations 
based on Brownian motion; see for example [3]. 

Note that, as it is only the u-left limit G(ui, u—, Z u _) which enters into @, 
there is no loss of generality to assume that G(w,m, Z) is left continuous in u 
for each oj and Z. Note also that as M is a semimartingale, Z is cadlag and 
adapted (see Thm 4.31]). 

We assume the existence of the left limits of Y. Hence, Y must have at most 
a countable number of discontinuities, and therefore it must be left-continuous 
at each t except possibly on a dt-mil\ set. Hence, if Y u satisfies ©, then so does 
Y 

J u— 1 

Z t + I F(u, Z u , Y u _)du + / [G(u-, Z u _) + Y u J\dM t = Q. 

J]t,T] J]t,T] 

Writing Y t * := Y t - we have a left-continuous process Y* which will also satisfy 
the desired equation, and therefore the writing of the left limits Y" t _ is unneces- 
sary (as we simply assume our solution is left-continuous). 
Given these arguments, we rewrite @ as 

Z t + I F(u,Z u ,Y u )du+ [ [G(u,Z u _) + Y u ]dM t = Q, 

J]t,T] J]t,T] 

3 A Martingale Representation Theorem 

Lemma 3.1. Any M. K valued martingale L defined on (tt, {^},P) can be rep- 
resented as a stochastic (in this case Stieltjes) integral with respect to the mar- 
tingale process M, up to equality P-a.s.. This representation is unique up to a 
d(M, M) t x P-null set. 

Proof. For i ^ j, (X u -, ei)(X u , ej) = 1 if and only if X jumps from e, to 6j at 
time u. Then 

(X u -,e l ){X u ,e j ) = (X u _,ei)(AX u ,ej) = (X u ^ 1 e l )(dX u ,e j ) 

and 

N? - / {X u _, ei ){dX u ,e ) 
J]o,t] 

is the number of jumps from to ej in ]0, t]. 
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From H|), dX t = A t X t dt + dM t , and so 

Aji{X u , ei}du + Q\\ 



o,t] 



where Ql J is the compensated jump martingale 



(X u -,ei}(dM u ,ej). 



Note that the filtration generated by X is the same as the filtration generated 
by the processes A™, i ^ j, 1 < i, j < N. Therefore, for any K.- valued square 
integrable {J~t} martingale L, we can write 



for some predictable processes 7^. (See [T] for a proof of this.) Define the 
predictable matrix process T by 



i — j, 



We can then write 



= L Q + I V 7 «(X u _ )ei )(dM u , ej -) 
J]o,t] 77 



L 



o,t] 



X* u _T u dM u . 



If we now consider an Wi K valued Tt martingale L, we can write this as 

{L\L\...,L K )\ 

where each term is an R- valued martingale. Hence we can write the L % term as 
a stochastic integral for some T z , and so the vector martingale can be written 



Lt — Lq 



where 



OA] 



v* pi 

u— u 
\* p2 

u— u 



u — u 



(3) 
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is a predictable M. KxN valued matrix process. 

Furthermore, this decomposition is unique, in the sense that if 



then 



L t = L + [ Y*dM u = L + [ Yl 
J]o,t] J]o,t] 

= f - Y 2 ]dM u =: S, 



t] 



l.t] 

P-a.s. By the Stieltjes product rule, 
= $*$ t * 

= <S>o$o+ [ $ u dM u [Y*-Y 2 }* + f [Y^-Y 2 ]dM u ^* u 
J]o,t] J]o,t] 

+ E [ F « - Y^]AM u AM:[Y^ - Y 2 }* 

0<u<t 

= Lt+ [ K - Y 2 ]d[M, A/yyJ - y u 2 ]* 

for some martingale L. This in turn implies that 

/ [yj - y„ 2 ]d(M, M) u [yJ - y 2 }* = o 

P-a.s., and so = F u 2 d(M, M) u x P-a.s. □ 

Lemma 3.2. For a square-integrable martingale L, the process Y satisfies the 
square integrability condition ]]^ < +°o dt-a.s. 

Proof. From Theorem [3l we have that 

L t = L + / Y u dM u 

J]0,t] 

and we also know that 

sup£:||L t || 2 < +oo. 
t 

We can express 

ll^tll 2 = ||Lo|| 2 + 2 f (L u ,Y u dM u )+ V ||y u AM„|| 2 
J P>t] <u<t 

and therefore 

E\\L t \\ 2 = E\\L \\ 2 + E \\YuAM u \\ 2 

0<u<t 

= E\\L \\ 2 + e[ \\Y u \\ 2 x Ju. 

J]0,t] 

Hence E Jj Q t j HYuH^dii < +oo, which implies that £ , ||y u ||? ft( < +oo di-a.s. 
as desired. □ 
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4 A Simple Case 

Lemma 4.1. Consider a simplified version of fH), namely 

Z t + { F(u, u)du + [ [G(lu, u) + Y u ]dM t = Q. (4) 
JH.T] Ju.t] 



]t,T] J]t,T] 

This equation has a unique solution 
Proof. First, let 



then 



Z t =E[Q- [ F(u)du\F t 

J]t,T] 

Z t - I F(u)du = E[Q - [ F{u)du\T t ] 

J]Q.t] J]O.T] 



i,T] 

is a square-integrable martingale (note the integrability assumptions on Q and 
F above), and so by Lemma 13. II has a representation 

F{u)du 

for some square-integrable predictable matrix process r u . Write Y u := T u - 
G(u), so 

Z t = f F{u)du+ f [G{u) + Y u ]dM u . 
J]o,t] J]o,t] 

By construction, Zt — Q, and a simple substitution argument gives 
Q = Z t + [ F{u)du+ [ [G{u) + Y u }dM t 

J]t,T] J]t,T] 

as desired. 

Furthermore, suppose (Z^Y^) and (Zf,Y?) both solve (g]). Then for all t 



Z\ - Z\ + [ [Yi - Y*]dM t = 0. 

J]t,T] 



]t,T] 

Taking an Tt conditional expectation shows that Z\ = P-a.s for each t. As 
Z 1 and Z 2 are cadlag this implies that they are indistinguishable [5l Lemma 
2.21]. Uniqueness from the martingale representation above then implies that 
Y 1 = Y 2 d(M, M) t x P-a.s. □ 

5 Increasing complexity 

We shall now reintroduce the variable Y u throughout our version of (J2), giving 
Z t + [ F(u, Y u )du + f [G(u) + Y u ]dM t = Q. (5) 

J]t,T] J t.T 
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Assume that F is Lipschitz continuous in the following way. There exists 
c G K such that for all u G [0, T] 

||F(u,yi) -F(u,Y„ 2 )|| 2 < ^minHYj - Y 2 \\ 2 er 
Note that this immediately implies 



E\\F(uX)-F(u,Y:)\\ 2 <c?E\\Yi-YZ\\ 



2 i i 2 



Lemma 5.1. Under the above Lipschitz condition, |5P has at most one solution, 
up to indistinguishability for Z and equality a.s. d(M,M) t x P for Y. 

Proof. Suppose (ZjX) and (Zf,Y t 2 ) are both solutions to ©. Then 

Z\ - Z 2 t = Z\ - Z% + / [F(u, Y$ - F(u, Y 2 )]du + f [Fj - F„ 2 ]dM t . 
J]o,t] J]o,t] 

Using the Stieltjes Chain rule for products, 

\\Z\ Z 2 \\ 2 = \\Z\ Zlf + 2 [ (Zl- ZlF(uX) - F(uX))du 

J]o,t] 

+ 2 [ (Zl- Zl X ~ Y*]dM u ) + W AZ I AZ t 

M*] Q<u<t 

and hence, taking expectations and evaluating at t = T, 
E\\Z\-Z 2 t f 

= -2/ E(Zl-ZlF(uX)-F{uX))du-E £ \\AZ] - AZ 2 \\ 2 

J\t,T\ t -~ 



2 1 1 2 



]t,T] 

-2 f E(Zl-ZlF{uX)-F{uX))du-E \X -Y^]AM. 

J]t,T] 

-2 [ E(Zl - Zl F{u, Yl) - F(u, Y 2 ))du 

J]t,T] 



t<u<T 



t<u<T 

EX-YZfxJu 



We now recall that, for any i£l and R inner product (, }, 

±2(o, 6) < -i|| a || 2 + x 2 ||6|| 2 

(simply expand < |] — =p x6|| 2 ). Using this, 
E\\Zl-Z\f 

= I [-2E(Zl - Zl,F{uX) - FiuX)) - FX - ^HxJ du 

J]t.T] 



< 



< 



l]t,T] 
f 

]t,T] 
f 

]t,T] 



-,E\\Z\ - Z 2 || 2 + x 2 E\\F(uX) - F(u, Y 2 )\\ 2 - EX - Y 2 \\ 2 Xv 



du 



-EX - Zlf + (x 2 c 2 - 1)EX ~ Y 2 \\x v 



da 
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and hence, through the use of an integrating factor, 



E\Z\ - Z\f < (x 2 c 2 -l) [ , 

J]t,T] 



e*'' E\\Y^-YXxJu. 



If we now set x = c _1 we see that E\\Z} — Z 2 \\ 2 — 0, indicating that Z\ = Z 2 
P-a.s. for each t. Right continuity then again implies that Z 1 and Z 2 are 
indistinguishable . 

If we set x < c , we conclude 



> / E\\Yt - Y*f Xu du = E [ Tr([Yj - Y 2 ]d(M, M) U [Y^ - Y 2 p 

J]t,T] J]t,T] 



and so Y 1 = Y 2 d{M, M) u x P-a.s. 

Therefore, we claim that the solutions (Z 1 , Y 1 ) and (Z 2 , Y 2 ) are equivalent, 
and consequently that any solution is unique (up to appropriate sets of measure 
zero). □ 

Lemma 5.2. Under the above Lipschitz condition, ^ has a solution. 

Proof. We now wish to demonstrate that for an arbitrary terminal condition 
Q, a solution (Z t ,Y t ) exists. We do so using a Picard-type iteration, where we 
define recursively (Z t n+1 , Y t n+1 ) to be the solution to 



z n+l 



+ f F(u, YS)du + f [G(u) + Y: +1 ]dM u = Q. 

J]t.T] J]t,T] 



This equation is of the type of ((3]), and so the existence of a unique solution 
is guaranteed by Lemma H. II We shall show that these iterates form a Cauchy 
sequence under an appropriate norm, and therefore that their limit exists and 
solves the desired equation. 
As earlier, we can show that 

E\\Z? +1 - Z2\\ 2 = -2 f E(Z n a +1 - ZZ,F(u,Y£) - F{u,Y^)du 

J]t.T] 



t,T] 
t,T] 



E\\Y: +i -Y:\\ 2 du 



and so for any i£l 
< e t / x ' 2 \\Z? +1 -Z™|| 2 



< 



(6) 

e-'- jh\\x-"- iu\\%au 

)t,T] J]t,T] 

Setting x < 2~ 1 l 2 c~ 1 we have 



/ e t '* 2 c 2 x 2 E\\Y: - Yr'W'xJu [ E\\Y^ - Y^Ju 

J]t,T] J]t.T] 



[ e^E\\Y^ - y:\\ 2 x Ju <\j e 2tc2 E\\Y: - Y^\\ x Ju 

J]t,T] 1 J]t,T] 

f 

]t,T] 



<2- n I e 2tc E\\Y* -YXx Ju 
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and so Y t n is a Cauchy sequence under an appropriate norm. By completeness, 
this implies that a limit exists. Considering again ([6]), we also see that Z r t l is 
a Cauchy sequence, and therefore again a limit exists. Furthermore, it is seen 
that these limits satisfy ©. □ 

6 A General Solution 

We now consider j2]) in full generality. We again shall assume Lipschitz con- 
tinuity on the generators F and G; in this case we shall require there to exist 
ceM such that for all u £ [0, T] 

\\F(u,ZlX)-FM,Y*)f < c 2 mm\\Y:-Y^\\l+c 2 \\Zl Z 2 J 2 
max ||G(u, Z\_) - G(u, Zl_)\\% < c 2 \\Zl_ - Z 2 U _\\ 2 . 

Again, these requirements are only needed to establish the weaker condition 

E\\F(u, Zl Yl) - F(u, Zl Y 2 )\\ 2 < c 2 E\\Y^ - Y 2 \\ 2 Xu + c 2 E\\Z\ Z*f 
E\\G{u,Zl_)-G{u,Zl_)\\\ u <c 2 E\\Zl_-Zl_\\\ 

We can further reduce the second of these to 

E\\G{u,Zl_) - G(u,Zl_)f < Zac 2 E\\Zl_ - Z\_f 

as ||.||^ < 3ci 1 1 . 1 1 2 , where a = max ue[0:T] maxjj |L4Jy|. 

We also note that as AZ U = Y U AM U and AM U — P-a.s. for all u, we know 
that 

E\\Zl_-Zl_f = E\\Zl-Zlf. 
Therefore, our assumption implies the condition 

E\\G(u, Z\_) - G(u, Zl_)f Xu < c 2 E\\Z l u Z 2 \\ 2 . 

Lemma 6.1. Under the above Lipschitz condition, has at most one solution 
up to indistinguishability for Z and equality a.s. d(M, M) t X P for Y . 

Proof. As above, suppose {Z\,Y^) and (Zf,Y 2 ) are both solutions to ([2|). 
Through the same calculations as before, we find that 

E\\Z\ - Z\f = -<l [ E{Z 1 u -Z 2 u ,F(u,ZlY^)-F(u,Z 2 u ,Y 2 ))du 
J]o,t] 

- I E\\G(u,Zi_)-G(u,Z 2 u _)+Yi-Y*\\ x Ju 
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and so 

E\\Z\ - Z 2 || 2 = / [-2E(Z l u - ZlF{u,ZlX) - F(u,Z 2 u ,Y 2 )) 

J]t,T] 

- E\\G(u, Zl_) - G(u, Zl_)f Xu - E\\Y* - Y 2 f Xu 

- 2E(G(u, Z\_) - G{u, Z 2 U _)X - Y 2 )x u ]du 

< I [\e\\ZI- Zlf + x 2 \\F(u,Z 1 u X)- F(u,ZlX)f 

J]t,T] X 

+ {\-l)E\\G{u,Zl_)-G{u 1 Zl_)f Xu 

y 

+ {y 2 -l)E\\Y^-Y 2 f Xu ]du 
for all x,y G.M.. Now let x = (2c) _1 and y = 1/2. In this case 

E\\Z\ ~Z 2 t f<\ [^E\\Zl - Zlf + -^\\F(u, Zl Y^) - F(u, Z 2 U1 y„ 2 )|| 2 
J]t,T] 4c 

+ 3E\\G(u, Z\_) - G{u, Zl_)f Xu - \e\YI - Y 2 f x Jdu 
< I \^E\\Z\ - Zlf + -E\\Y^ Y 2 fx u + \ml Zlf 

J]t,T] 4 4 

+ 2,0^1- Zlf - \ E \\Y^ - Y^f Xu ]du. 

Hence 

E\\Z\ - Zlf < (7c 2 + -) ( E\\Zi - Zlfdu 

4 J]t,T] 

and so an application of Gronwall's lemma implies 

E\\Z\ - Z 2 || 2 = 0, 

i.e. Zl = Zl P-a.s. for each t. Right continuity again implies Z 1 and Z 2 are 
indistinguishable . 

From the above we can also deduce 

/ E\\Y^-Y 2 f x Ju<2(7c 2 + \) [ E\\Zl- Zlfdu = 

J]t,T] 4 J]t,T] 

and therefore Y 1 = Y 2 d(M,M) t x P-a.s. Hence the solution is once again 
unique. □ 

Theorem 6.2. Under the above Lipschitz condition, fij^) has a solution. This so- 
lution is then unique up to indistinguishability for Z and equality a.s. d(M, M)t x 
P for Y. 

Proof. Once again, we shall do this using a Picard-type iteration. We define 
recursively (Z n+1 ,Y n+1 ) to be the solution of 

zr 1 + I f(u, z:,Y: +i )du + f g( u , zi_) + Y: +i dM u = q. 

J]t,T] J]t,T] 



11 



Each of these iterates is guaranteed to exist and be unique by Lemmas 15.21 and 
Using the same procedure as above we obtain 



2 



^ (7) 



<(4c 2 + l) / [E\\Z^- Z:f+E\\Z:-Z^f]du. 

J]t,T] 

This implies 

E\\Zr l - Z?f < (4c 2 + 1) / [E\\ZZ +1 Z»|| 2 + E\\Zl Z^f\du. 

J]t,T] 

Rearrangement and integration gives 

/ E\\ZZ +1 -ZZfdu< (4c 2 + l) / e (4c 2 +i)(u-t) /" E\\Z n s -Z n s - x fdsdu 

J]t,T] J]t,T] J]u,T] 

and hence 

E\\Z^ - Zlfdu < [r(4c2 + 1) f (4C2+1)] " / E\\Z\ Zlfdu. 

t,T] n - J]t,T] 

From this, we know that {Z™} forms a Cauchy sequence, except possibly on some 
<it-null set, and so the P-a.s. limit exists by completeness and right continuity. 
Similarly we can then rearrange ([7]) to give 

/ E\\Y: +1 -Y:\\\Ju < 2(4c 2 + l) / EWZ^-ZZf+EWZZ-Z^lfdu, 

J]t,T] J]t,T] 

again showing that {l" n } forms a Cauchy sequence, and so its limit exists. 
Again, these limits can be seen to satisfy ©. The desired uniqueness properties 
follow from Lemma lo\Tl □ 



7 Conclusion 

We have shown here that for an equation of the form of @ , there exists a square 
integrable solution (Z, Y) , where Z is an M. K valued adapted process and Y is 
an ~§L KxN valued adapted process. We have shown that, and without loss of 
generality, Y can be taken to be left-continuous. We have also shown that thise 
solution is unique up to P-indistinguishability for Z and equality d(M, M) u x P- 
a.s. for Y. 
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